Word Association Spaces for Predicting Semantic Similarity Effects in Episodic Memory

A common assumption of theories of memory is that the meaning of a word can be represented by a vector which places a word as a point in a multidimensional semantic space (e.g. Landauer & Dumais, 1997; Burgess & Lund, 2000; Osgood, Suci, & Tannenbaum, 1957). Representing words as vectors in a multidimensional space allows simple geometric operations such as the Euclidian distance or the angle between the vectors to compute the semantic (dis)similarity between arbitrary pairs or groups of words. This representation makes it possible to make predictions about performance in psychological tasks where the semantic distance between pairs or groups of words is assumed to play a role. One recent framework for placing words in a multidimensional space is Latent Semantic Analysis or LSA (Derweester, Dumais, Furnas, Landauer, & Harshman, 1990; Landauer & Dumais, 1997; Landauer, Foltz, & Laham, 1998). The main assumption is that the similarity between words can be inferred by analyzing the statistical regularities between words and text samples in which they occur. For example, a textbook with a paragraph that mentions “cats” might also mention “dogs”, “fur”, “pets” etc. This knowledge can be used to infer that “cats” and “dogs” are related in meaning. The technique underlying LSA is singular value decomposition (SVD). This procedure is applied to the matrix of word-context frequencies in a high dimensional space (typically with 200-400 dimensions) in which words that appear in similar contexts are placed in similar regions of the space. Interestingly, some words that never occur in the same context might still be similar in LSA space if they co-occurred with other words that do occur together in text samples. Landauer and Dumais (1997) applied the LSA approach to over 60,000 words appearing in over 30,000 contexts of a large encyclopedia. More recently, LSA was applied to over 90,000 words appearing in over 37,000 contexts of reading material that an English reader might be exposed to from 3 grade up to 1 year of college from various sources such as textbooks, novels, and newspaper articles. The LSA representation has been successfully applied to multiple choice vocabulary tests, domain knowledge tests and content evaluation (see Landauer & Dumais, 1997; Landauer et al. 1998). In this research, we will apply scaling techniques such as SVD as well as Multidimensional Scaling on a large database of free association collected by Nelson, McEvoy, and Schreiber (1999) containing norms for first associates for over 5000 words. By applying scaling methods on the free association norms, we hope to uncover the latent information available in the free association norms that is not directly available by investigating simple measures for associative strengths based on the direct and indirect associative strengths through short chains of associates (e.g., Nelson & Zhang, 2000). The basic approach is illustrated in Figure 1. The free association norms were represented in matrix form with the rows representing the cues and the columns representing the responses. The entries in the matrix are filled by some measure of associative strength between cues and responses. By applying scaling methods on the matrix, words are placed in a high dimensional space such that words with similar associative patterns are placed in similar regions of *Send correspondence to: Mark Steyvers, Department of Cognitive Sciences, 3151 Social Sciences Plaza, University of California, Irvine, CA 92697-5100. [email protected] the space. We will refer to the resulting space as the